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Introduction 


Beyond federal requirements in the Every Student Succeeds Act and state-level accountability 
systems, school districts across the country have an essential role in school-level 
accountability, especially assessing student learning and wellness. Today — with unfamiliar 
school configurations and unknown impacts on student outcomes — districts can adapt how 
they measure school quality to better understand how schools are operating and how 
students are learning. 


Numerous school districts have developed School Performance Frameworks (SPFs) with 
shared metrics for quality among their schools. With a missing year of data on state 
assessments, widespread adoption of distance learning, and a renewed emphasis on student 
health and safety, these districts must adjust their SPFs to account for missing data and 
shifting priorities. 


For those districts that have yet to establish SPFs, the urgent need to understand what schools 
are doing and how it affects student outcomes can serve as a Call to action. SPFs are an 
essential tool that can help districts set priorities for their schools and support them as 
they continue to adapt and improve. Moreover, unlike federal- and state-level officials, school 
district leaders are well-positioned to establish metrics that are responsive to local 
circumstances. 


Bellwether has published numerous resources in recent weeks and months about the role of 
metrics and accountability during the pandemic; this deck is a tool for district leaders as they 
develop and adapt their approaches. 
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Districts need to rethink their approaches to measuring 
student learning and wellness 


The COVID-19 pandemic, school closures, advent of distance learning, and 
suspension of state testing have disrupted how districts assess and manage their 
systems of schools. Districts must adapt their approaches accordingly. 


Districts must shift how they assess school quality, support continuous 
improvement, and hold schools accountable. 


It is essential for districts to understand how well schools are meeting the needs of 
students, including not only academic growth but also student wellness. 


Districts managing systems in which foundational decisions affecting student learning sit with 
building leaders need to continue strong oversight and support decision-making at the 
school and system level. 


Districts have a responsibility to protect student interests while being mindful of the burdens 
they place on school leaders and educators. 


Measures of school quality are essential for understanding equity and ensuring that the 
system of schools is collectively meeting the needs of all students. 


The data gaps created by lack of testing fundamentally disrupted the ways in which districts 
have assessed schools’ academic performance. 
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The need to rethink these measures has occurred within a 
broader discussion of accountability 


Current circumstances present challenges for districts in managing school quality, as well as a 
window to refresh their approach. Districts with SPFs must quickly and nimbly adjust them, 
as well as how they are used for accountability and continuous improvement. 


At the same time, all districts — regardless of whether they have SPFs — can start now to build 
a foundation for how schools and systems recover, how we assess what we value about 
school performance, and how to better support students to thrive moving forward. 


Some have argued that we should forgo accountability for the last year and beyond, while 
others believe we should return to past accountability practices as soon as possible. 


Neither response is adequate. Rather, districts must develop new and better approaches to 
assess student learning, school performance, and quality. 


Districts need not start from scratch; many can evolve their existing SPFs to assess and 
communicate school quality. 


Adaptations shouldn’t narrowly focus on addressing current gaps in testing data. 


In developing and evolving their SPFs, districts have an opportunity to innovate in ways that 
are responsive to the needs of their local communities. 
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To evolve SPFs or define new locally developed measures 
and goals, districts must answer several key questions 


What are the key concepts or indicators that make up the district’s current/past approach to 
assessing school quality? Are there other concepts or indicators related to school performance that 
become newly or more important for districts to assess in a COVID-19 world? 


@ How has the district measured school performance within these indicators? 
@ Why does the district focus on these measures? What questions is the district trying to answer? 


What kinds of decisions does this data inform? What are the stakes of these decisions for 
students, schools, communities, and districts? 


What problems or complications may prevent the district from using these measures now? Are 
there other limitations of those measures that the district is concerned about or aware of? 


What are other ways the district could get at the questions that these measures are intended to 
help us answer? What information is the district currently collecting from schools? Is there 
additional information that schools are collecting that might help? Are there other information 
sources that would be useful that districts and schools don’t currently have but potentially could? 


What steps must the district take to implement these other ways of assessing quality or use them 
well? Are there any statutory or regulatory constraints? How will the district engage stakeholders 
and build buy-in for the changes? 
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This toolkit walks through these questions to support 
adjustments to measuring student learning and wellness 


To adapt existing school performance frameworks during the pandemic, 
districts must audit the measures they have previously used, assess limitations 
of those measures in the current context, identify other measures that are 
newly relevant, and develop an approach to answering key questions about 
student learning and well-being. 


Crucially, a variety of stakeholders use school performance frameworks to 
make decisions for their families, schools, and communities. Districts must 
prioritize providing information that is most useful to those stakeholders. 
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school performance frameworks typically include measures 
that align to six domains (slide 1 of 2) 


1. Student Growth/Progress 


e State standardized 
assessments: 
Criterion-referenced growth, 
median growth percentile, 
student growth percentile, 
value-added measures 


e Subgroup growth 


2. Student Achievement 
(Proficiency) 


State standardized 
assessments: percent of 
students proficient, proficiency 
rates relative to other schools 
or state average 


e Subgroup proficiency 


This toolkit will walk through key 
questions for districts one by one for the 
first domain, followed by a summary of 

each of the other five domains. 


The measures included here are examples and do 


not reflect all possible measures for each domain 


3. Postsecondary Readiness 


Graduation rate/Adjusted Cohort 
Graduation Rate (ACGR) 


Credit units: 9th- or 10th-grade 
students on-track to graduate 


ACT/SAT: participation rates; 
percent scoring above a certain 
threshold; average score compared 
to national averages 


Completion of AP/IB or postsec 
credits while in high school 


Completion of industry-recognized 
credentials 


Postsecondary enrollment: College 
acceptance rate; percent of 
graduates enrolled in 
postsecondary; remediation rate for 
graduates attending postsecondary 


Post-graduation employment: 
Percent of graduates employed in 
the fall following graduation 


school performance frameworks typically include measures 


that align to six domains (slide 2 of 2) 


5. Student & Family 


4. Model-Specific Goals Engagement & Well-Being 


e Extent to which school Instructional minutes 
implements/students complete 
practices aligned with specific 

school model (e.g., internships 
or capstones; volunteering 


hours) 


Average daily attendance 
e Chronic absenteeism rates 
e Truancy rates 


e Valid and reliable measures of | @ Reenrollment rates 


student learning in subjects 
other than ELA and math (e.g., 
additional languages for 
bilingual schools; science) 


e School climate measures 
e Parent surveys 


_e@ Student surveys 
e Portfolios or other tools to 


demonstrate student learning | ® Supports for emotional, 


behavioral, mental, and physical 
health 


This toolkit will walk through key questions 
for districts one by one for the first domain, 


followed by a summary of each of the other 
five domains. 


6. Opportunity Gaps 


Exclusionary discipline rates 


Exclusionary discipline rates for 
student subgroups 


Student mobility 

Special education delivery 

Racial and economic segregation 
Teacher demographic data 
Teacher-student match data 


Availability & enrollment in 
advanced, rigorous coursework; 
gifted and talented programs; 
courses in arts, social sciences, 
sciences, and technology; 
formalized systems of tutoring and 
other academic supports 


e Enrollment in above by subgroup 


The measures included here are examples and do 
not reflect all possible measures for each domain 
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Domain Example: Student Growth/Progress 


Districts can use this process to adapt existing SPFs or 
define new school measures and goals 


The following slides sequentially walk through the questions districts must answer to 
adapt their school performance framework or define new locally developed school 


measures and goals (see here for the complete list of questions). 


Each slide focuses on one of these key questions, provides instructions for examining 
practices and priorities to answer that question, and identifies key considerations. 


This set of slides illustrates how a district might go through this process, using the 
student growth/progress domain and the state standardized assessments 
measure as an example. A district should complete this same process for all 
measures it is considering adapting or creating. 


A workbook available here was developed to answer similar questions 
for charter school authorizers, and may also be useful for districts. 
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@® What domains make up the district's current/past 
approach to assessing student growth/progress? 


The district must identify the What domains make up the 


domains that make up its district's current/past approach Student growth/progress 
to assessing school quality? 


vision for school quality 


e Review the domains in the 
district's school 
performance framework 


e Identify the domains that 
the district focuses on in 
measuring school 
performance 


e Identify which of these 
domains are common 
across schools and which 
are differentiated for 
certain types of schools 
(e.g., by grade level) or for 
individual schools 


The district must identify 
how it measures these 
domains of school quality. 


For each of the domains 
identified, the district must 
list out the measures that 
it previously used, or could 
use, to assess school 
performance for that 
domain 


This list may get long; 
there will likely be multiple 
measures for each domain 


The district should not 
pare down the list of 
measures at this point, but 
consider this a brainstorm 


This list of measures is the 
district’s initial list of 
student growth/progress 
measures 


How has the district measured school performance in 
student growth/progress? 


What domains make up the district’s 


current/past approach to assessing Student growth/prog ress 
school quality? 


How has the district measured school State standardized assessments (e.g., criterion- 
performance in these domains? referenced growth, median growth percentile, student 
growth percentile, value-added measures) 


BELLW:ETHER 


EDUCATION| PARTNERS 12 


3 Why does the district focus on these measures? What 
questions is the district trying to answer? 


The district must be clear 
about the rationale for 
including these measures in 
its school performance 
frameworks 


e For each of the student 
growth/progress 
measures, the district must 
think through why the 
measure is important 


e Given that rationale, the 
district must clarify the 
question it is trying to 
answer with that measure 


Consider. Are any questions 
answered by multiple 
measures? Are each of the 
measures worth collecting 
or necessary to answer the 
question? 


What domains make up the 
district’s current/past approach 
to assessing school quality? 


How has the district measured 
school performance in these 
domains? 


Why does the district focus on 
these measures? 


What questions is the district 
trying to answer? 


Student growth/progress 


State standardized assessments: (e.g., criterion-referenced growth, 
median growth percentile, student growth percentile, value-added 
measures) 


Student growth or learning gains reflect how well a school is 
supporting children’s learning; growth or gains provide a more 
accurate picture of a school’s contribution to children’s learning than 
achievement data alone 


Are students learning what we expect them to learn over the course of 
the year? Are students progressing at least one grade level over the 
course of the year? Are students on track to reach grade-level 
expectations? 
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4) What kinds of decisions do these data inform? 


Student learning and 
wellness measures are 
ultimately used to make 

decisions about schools. 


e The district must identify 


the types of decisions 
that these data have 
historically informed 


e There will likely be 
substantial overlap in 
the types of decisions 


Consider. How do teachers 
and school leaders use this 
information to inform their 
instructional approaches 
and priorities? How do 
families and the community 
use this information to 
understand school quality 
and make decisions for 
students? 


What domains make up the district’s 
current/past approach to assessing 
school quality? 


How has the district measured 
school performance in these 
domains? 


Why does the district focus on these 
measures? 


What questions is the district trying 
to answer? 


What kinds of decisions do these 
data inform? 


Student growth/progress 


State standardized assessments: (e.g., criterion-referenced growth, 
median growth percentile, student growth percentile, value-added 
measures) 


Student growth or learning gains reflect how well a school is 
supporting children’s learning; growth or gains provide a more accurate 
picture of a school’s contribution to children’s learning than 
achievement data alone 


Are students learning what we expect them to learn over the course of 
the year? Are students progressing at least one grade level over the 
course of the year? Are students on track to reach grade-level 
expectations? 


Ratings shared with parents and public; school improvement planning; 
decisions about driving systemic improvement 


© What problems or complications may prevent the 
district from using those measures now? 


The district must anticipate 
complications with 
relying on these measures 
to inform decisions. 


e For each of the student 
learning and wellness 
measures, the district must 
identify potential 
challenges 


e Key questions may help 
elucidate challenges: What 
is the short- and long-term 
availability of the data? 
Does the district question 
or mistrust the validity of 
the data for this measure? 


Consider. Are there any 
actions the district could take 
to solve these problems? Do 

any measures have 
insurmountable challenges? 
If so, is it crucial to continue 
using those measures, at 
least at this point in time? 


What domains make up the 
district’s current/past approach 
to assessing school quality? 


How has the district measured 


school performance in these 
domains? 


Why does the district focus on 
these measures? 


What questions is the district 
trying to answer? 


What kinds of decisions do these 
data inform? 


Student growth/progress 


State standardized assessments: Criterion-referenced growth, median 
growth percentile, student growth percentile, value-added measures 


Student growth or learning gains reflect how well a school is supporting 


children’s learning; growth or gains provide a more accurate picture of a 
school’s contribution to children’s learning than achievement data alone 


Are students learning what we expect them to learn over the course of 


the year? Are students progressing at least one grade level over the 
course of the year? Are students on track to reach grade-level 
expectations? 


Ratings shared with parents and public; school improvement planning; 
decisions about driving systemic improvement 


What complications may prevent 


the district from using those 
measures now? 


States did not administer state standardized assessments in spring 


2020, so growth data are not available for this year or next year, and 
potentially further into the future 


oO What other ways could the district get at the questions 
these measures are intended to help us answer? (1 of 2) 


There may be viable 
alternatives to the data 
historically used for 
this measure. 


e For each of the measures 
with prohibitive challenges, 
the district must brainstorm 
data sources or measures 
that could answer the same 
question 


e The districts must prioritize 
data that fill a gap created 
by COVID-19 and focus on 
data that the district and 
schools already collect or 
could be collected with 
limited burden 


Consider. Are there steps the 
district should take to collect a 
new source of data in the future? 

Are there partners the district 
can reach out to or permissions 
the district must secure in order 
to have access to these data in 

the long term? 


What domains make up the 
district’s current/past approach to 
assessing school quality? 


How has the district measured 
school performance in these 
domains? 


Why does the district focus on 
these measures? 


What questions is the district trying 
to answer? 


What kinds of decisions do these 
data inform? 


What complications may prevent 
the district from using those 
measures now? 


What are other ways the district 
could get at the questions that 
these measures are intended to 
help it address? 


Student growth/progress 


State standardized assessments: Criterion-referenced growth, median | 


growth percentile, student growth percentile, value-added measures 


Student growth or learning gains reflect how well a school is 


supporting children’s learning; growth or gains provide a more 
accurate picture of a school’s contribution to children’s learning than 
achievement data alone 


Are students learning what we expect them to learn over the course of 


the year? Are students progressing at least one grade level over the 
course of the year? Are students on track to reach grade-level 
expectations? 


| Ratings shared with parents and public; school improvement planning; | 


decisions about driving systemic improvement 


States did not administer state standardized assessments in spring 
2020, so growth data are not available for this year or next year, and 
potentially further into the future 


*See next slide 


oO What other ways could the district get at the questions 
these measures are intended to help us answer? (2 of 2) 


There may be viable What domains make up the district’s 
alternatives to the data current/past approach to assessing 
' ' itv? 
historically used for ecneelaualty 
this measure. Odections tes 
e For each of the measures 
with prohibitive challenges, What are other ways the district 
the district must brainstorm could get at the questions that these 
data sources or measures measures are intended to help it 
address? 
that could answer the same 
question 


e The districts must prioritize 
data that fill a gap created 
by COVID-19 and focus on 
data that the district and 
schools already collect or 
could be collected with 
limited burden 


Consider. Are there steps the 
district should take to collect a 
new source of data in the future? 

Are there partners the district 
can reach out to or permissions 
the district must secure in order 
to have access to these data in 


Student growth/progress 


*See previous slide 


Benchmark or interim measures commonly used by schools (e.g., 
NWEA adapted to produce a summative score, ANet benchmarks) 


Student progress data generated by online/tech-based learning 
programs (e.g., DreamBox, Waterford) 


Curriculum-based assessments used by schools 
Formative assessments used by schools 
Students’ progress through independently defined levels of 


performance (e.g., Lexile or grade-level progress in reading, 
executive functioning levels for adult learning) 
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@ What steps must the district take to implement these 
other ways of assessing quality or using them well? 


The district must plan 
carefully when revising its 
approach 


e The district must identify 
statutes or regulations 
that affect how it can adapt 
school performance 
frameworks (e.g., school 
board policies) 


e The district must identify 
stakeholders who need to 
have input and buy-in for 
changes to school 
performance frameworks 
(e.g., board members, 
school leaders, families) 


Consider. Given the number of 
changes schools and 
communities are navigating, 
careful planning and strong 
communication are of utmost 
importance 


What domains make up the 


district’s current/past approach to 


assessing school quality? 


Questions 2 - 6 


What steps must the district take 
to implement these other ways of 
assessing quality or use them 
well? 


Student growth/progress 


*See previous slide 


Assess constraints in statutes, regulations, or local policies in 
consultation with local school board and state education agency, as 
needed 


Engage school leaders to understand what assessments they use, 
and how they use them (and other sources of data) to identify and 
address areas of continuous improvement 


Engage parent-teacher associations or other parent groups to 
understand how data on student growth/progress informs public 
understanding and family decision-making 


Communicate with school board members and other public officials 
about the ways in which the district plans to change the domains, to 
solicit input and build buy-in 


Determine ambitious but achievable expectations for students and 
schools 
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Domain Summaries: 


Student Growth/Progress (see previous slides) 
Student Achievement/Proficiency 
Postsecondary Readiness 

Model-Specific Goals 

Student & Family Engagement & Well-Being 
Opportunity Gaps 


Student Achievement/Proficiency: Summary of Key 


Questions (1 of 2) 


What domains make up our current/past approach to assessing student achievement/proficiency? 


How has the district measured school 
performance in these domains? 


Why does the district focus on these measures? 


What questions is the district trying to answer? 


What kinds of decisions do these data inform? 


What complications may prevent the district from 
using those measures now? 


What are other ways the district could get at the 
questions that these measures are intended to 
help it address? 


Subgroup proficiency on state assessments 


Students need to master grade-level content in order to be prepared for success in 
the next level; holding schools accountable for whether students are meeting 
common grade-level standards is important as a matter of equity to ensure high 
expectations for all students 


Are there differences in performance between student subgroups? Is this school 
improving outcomes for underserved subgroups? 


Ratings shared with parents and public; school improvement planning; decisions 
about driving systemic improvement 


States did not administer state standardized assessments in spring 2020, so these 
data are not available 


Benchmark or interim measures commonly used by schools (e.g., NWEA adapted to 
produce a summative score, ANet benchmarks) 


Student progress data generated by online/tech-based learning programs (e.g., 
DreamBox, Waterford) 


Curriculum-based assessments used by schools 
Formative assessments used by schools 


Students’ progress through independently defined levels of performance (e.g., Lexile 
or grade-level progress in reading, executive functioning levels for adult learning) 
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Student Achievement/Proficiency: Summary of Key 
Questions (2 of 2) 


What domains make up our current/past approach to assessing school quality? 


Assess constraints in statutes or regulations, in consultation with local school board 
and state education agency, as needed 


Audit the validity and reliability of existing assessments and determine for what types 
of accountability decisions it is appropriate to use them 


Engage school leaders to understand what assessments they use, and how they use 


What steps must the district take to implement them (and a sources of data) to identify and address areas of continuous 
improvemen 


these other ways of assessing quality or use 


them well? Engage parent-teacher associations or other parent groups to understand how data 


on student growth/progress informs public understanding and family decision-making 


Communicate with school board members and other public officials about the ways 
in which the district plans to change the domains, to solicit input and build buy-in 


Determine ambitious but achievable expectations for students and schools 
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Postsecondary Readiness: Summary of Key Questions 


What domains make up our current/past approach to assessing postsecondary readiness? 


How has the district measured school 
performance in these domains? 


Why does the district focus on these measures? High school graduation is an important outcome that predicts a host of later 
economic and life outcomes 
What questions is the district trying to answer? a aa in this school persisting to graduation and graduating at acceptable 


Ratings shared with parents and public; school improvement planning; decisions 
about driving systemic improvement 


Graduation rate/Adjusted Cohort Graduation Rate (ACGR) 


What kinds of decisions do these data inform? 


What complications may prevent the district Students whose education was disrupted due to COVID-19 may have lower on-time 
from using those measures now? graduation rates for several years into the future 


What are other ways the district could get at the 
questions that these measures are intended to 
help it address? 


Calculate additional graduation rates over extended timelines (e.g., 5-year, 6-year) to 
account for COVID-related disruptions 


Analyze and take steps to mitigate potential unintended consequences (e.g., 
whether adjusted graduation rates will be perceived as lowering standards) 


What steps must the district take to implement Engage parent groups, business leaders, and postsecondary institutions to 
these other ways of assessing quality or use understand how graduation data inform their decisions 
them well? 


Solicit input and build buy-in with school board members and other public officials 


Determine ambitious but achievable expectations for students and schools 
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Model-specific goals: Summary of Key Questions 


What domains make up our current/past approach to assessing model-specific goals? 


How has the district measured school 
performance in these domains? 


Why does the district focus on these 
measures? 


What questions is the district trying to 
answer? 


What kinds of decisions do these data 
inform? 


What complications may prevent the district 
from using those measures now? 


What are other ways the district could get at 
the questions that these measures are 
intended to help it address? 


What steps must the district take to 
implement these other ways of assessing 
quality or use them well? 


Extent to which school implements/students complete practices aligned with the school model 
(e.g., internships or capstones; volunteering hours) 


Districts can create flexibility for schools to create specialized educational programs that 
advance specific models or goals (e.g., bilingualism, arts, social justice). Districts that oversee 
these systems have a responsibility to assess whether such schools are implementing the 
models with fidelity 


Is this school implementing its model as intended? 


Ratings shared with parents and public; school improvement planning; decisions about driving 
systemic improvement 


Safe reopening or extended distance learning may require some changes in schools’ models 
or how they implement associated activities 


Are there measures that some schools collect as part of their model that are more important 
now and should be a higher priority to collect for all schools/students (e.g., measures of 
student well-being or health)? 


Engage school leaders to identify existing and potential measures that capture the specific 
value and outcomes of their school model 


Engage parents to understand what measures would be useful in understanding a school’s 
implementation of a specific model 


Solicit input and build buy-in with school board members and other public officials about the 
importance of different school models and tailored measures of quality 


Determine ambitious but achievable expectations for students and schools 


Student and family engagement and well-being: 
Summary of Key Questions 


What domains make up our current/past approach to assessing student and family engagement 
and well-being? 


How has the district measured school : 
. : Average daily attendance 
performance in these domains? 


Why does the district focus on these 
measures? 


What questions is the district trying to answer? | Are students attending school regularly? 


Ratings shared with parents and public; school improvement planning; decisions about 
driving systemic improvement 


Students need to be participating in the learning environment in order to grow and advance 


What kinds of decisions do these data inform? 


What complications may prevent the district Unclear how to meaningfully define or measure attendance in a distance learning context; 
from using those measures now? short-term closures and illness are likely to elevate absenteeism rates 


What are other ways the district could get at Measures of engagement with online learning platforms (e.g., logins, presence during 
the questions that these measures are intended | synchronous instructional time, software-based engagement tracking, completion of daily 
to help it address? objectives or activities) 


Analyze and take steps to mitigate potential unintended consequences (e.g., how 
attendance data informs how districts allocate and schools receive resources) 


What steps must the district take to implement | Engage parent groups to understand how they can or need to be engaged with attendance 
these other ways of assessing quality or use during distance learning 
them well? 


Solicit input and buy-in from school board members and other public officials 


Determine ambitious but achievable expectations for students and schools 


Opportunity gaps: Summary of Key Questions 


What domains make up our current/past approach to assessing equity? 


How has the district measured school 
performance in these domains? 


Why does the district focus on these 
measures? 


What questions is the district trying to 
answer? 


What kinds of decisions do these data 
inform? 


What complications may prevent the 
district from using those measures now? 


What are other ways the district could get 
at the questions that these measures are 
intended to help it address? 


What steps must the district take to 
implement these other ways of assessing 
quality or use them well? 


Exclusionary discipline rates for student subgroups 


Higher rates of exclusionary discipline for some students may reflect educator biases and/or 
exacerbate disparities in other outcomes 


Are there disparities in the rate at which different student subgroups experience exclusionary 
discipline? 


Ratings shared with parents and public; school improvement planning; decisions about driving 
systemic improvement 


Unclear how to meaningfully define or measure exclusionary discipline rates in a distance 
learning context 


Unclear until schools develop norms around discipline policies and practices in virtual 
environments 


Engage parent groups to understand what data could replace discipline rate data and be 
useful for decision-making 


Solicit input and buy-in from school board members and other public officials, as well as those 
affected by the decisions (e.g., school leaders, counselors, families) 


Determine ambitious but achievable expectations for students and schools 
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